The present invention relates to a filter bank approach to adaptive filtering method using independent component analysis. More particularly, the invention relates to a method of improving the performance of adaptive filtering method by applying independent component analysis that is capable of reflecting the secondary or even higher order statistical characteristics to adaptive filtering algorithm using the filter bank approach.
Independent component analysis is capable of separating default signals by receiving input signals of mixed sound sources from unknown channels and restoring mutually independent sound source signals.
Let us assume that N number of probabilistically independent sound sources s(t)=[s1(t), s 2(t), . . . , sN(t)]T are mixed through arbitrary channels and the sound sources are measured by N number of sensors.
In this instance, if the channels could be modelled by a simple addition of weighting factors, then the measured signals x(t)=[x1(t), x2(t), . . . , xN(t)]T by the sensors could be represented as follows.x(t)=A·s(t)  [Mathematical Equation 1]
where, A is an unknown invertible matrix which is known as mixed matrix.
Hence, the problem here is to obtain a reverse matrix of the mixed matrix A in order to restore the sound source signals only using the sensor signals that can be measured.
However, from the perspective of the sound source signal separation, the process of obtaining a restored signal whose original sequence of the sound source signal is reversed or a restored signal whose magnitude differs significantly from the sound source signal, would not cause a serious problem since this process does not affect the waveform itself.
Consequently, a restored signal u(t) whose magnitude and sequence are different from the sound source signal is obtained from the following mathematical equation by deducing a separation matrix W which permits the above process.u(t)=W·x(t)  [Mathematical Equation 2]
In this instance, in order to deduce the separation matrix W, an assumption which makes the sound sources to be mutual independent is established. This means that a signal originating from one sound source dose not interfere with another signal which is originating from another sound source. This is a reasonable assumption to be made in the real world and this statistical independence is inclusive of all the statistical characteristics for any order.
Also, since the statistical independence is not related to magnitude and sequence, the separation matrix W in Mathematical Equation 2 can be obtained. The separation matrix could be trained by using the following training method which maximizes the statistical independence among the sound source signals.
                                          Δ            ⁢                                                  ⁢            W                    ∝                                                    [                                  W                  T                                ]                                            -                1                                      -                                          φ                ⁡                                  (                  u                  )                                            ⁢                              x                T                                                    ,                                  ⁢                              φ            ⁡                          (                                                u                  i                                ⁡                                  (                  t                  )                                            )                                =                      -                                                                                ∂                                          P                      ⁡                                              (                                                                              u                            i                                                    ⁡                                                      (                            t                            )                                                                          )                                                                                                  ∂                                                                  u                        i                                            ⁡                                              (                        t                        )                                                                                                              P                  ⁡                                      (                                                                  u                        i                                            ⁡                                              (                        t                        )                                                              )                                                              .                                                          [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          3                ]            
where, P(ui(t)) represents a value which approximates the probability density function of the sound source signal ui(t) which is to be deduced.
A mixed signal, which is a simple addition of weighting factors, is hard to find in the real world, instead the sensor receives a mixed signal that is a convolution type mixture between time delays due to the transmission and surrounding environment. The convolution type mixed signal could be represented as follows.
                                          x            i                    ⁡                      (            t            )                          =                              ∑                          j              =              1                        N                    ⁢                                    ∑                              k                =                0                                            K                -                1                                      ⁢                                                            a                  ij                                ⁡                                  (                  k                  )                                            ⁢                                                s                  j                                ⁡                                  (                                      t                    -                    k                                    )                                                                                        [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          4                ]            
where, xit) represents measured signals, sj(t) represents N number of sound sources and a ij(k) represents mixed filter coefficients for the length K.
In order to separate the sound source signal, a feedforward filter structure in FIG. 1 or a feedback filter structure in FIG. 2 could be utilized.
In the feedforward filter structure, a restored signal ui(t) could be expressed as follows.
                                          u            i                    ⁡                      (            t            )                          =                              ∑                          j              =              1                        N                    ⁢                                    ∑                              k                =                0                            K                        ⁢                                                            w                  ij                                ⁡                                  (                  k                  )                                            ⁢                                                x                  j                                ⁡                                  (                                      t                    -                    k                                    )                                                                                        [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          5                ]            
where, wij(k) is filter coefficients for restoring the original sound source signal.
In this structure, the corresponding training methods for the coefficients wii(0) with no time delay between input and output and the coefficients wii(k), k≠0 with a time delay between the input and output could be expressed as follows.
                                          Δ            ⁢                                                  ⁢                          W              ⁡                              (                0                )                                              ∝                                                    [                                                      W                    ⁡                                          (                      0                      )                                                        T                                ]                                            -                1                                      -                                          φ                ⁡                                  (                                      u                    ⁡                                          (                      t                      )                                                        )                                            ⁢                                                x                  ⁡                                      (                    t                    )                                                  T                                                    ,                                  ⁢                              Δ            ⁢                                                  ⁢                                          w                ij                            ⁡                              (                k                )                                              ∝                                    -                              φ                ⁡                                  (                                                            u                      j                                        ⁡                                          (                      t                      )                                                        )                                                      ⁢                                          x                j                            ⁡                              (                                  t                  -                  k                                )                                                    ,                                  ⁢                              φ            ⁡                          (                                                u                  i                                ⁡                                  (                  t                  )                                            )                                =                                                    ∂                                  P                  ⁡                                      (                                                                  u                        i                                            ⁡                                              (                        t                        )                                                              )                                                                              ∂                                                      u                    i                                    ⁡                                      (                    t                    )                                                                                      P              ⁡                              (                                                      u                    i                                    ⁡                                      (                    t                    )                                                  )                                                                        [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          6                ]            
where, w(0) is a matrix consisting of the coefficients with no time delay wij(0) and each of u(t) and x(t) represents vectors consisted of ui(t) and xi(t) respectively.
The coefficients wij(0) with no time delay control the size of data in order to maximize the amount of information which is transferred through a nonlinear function. The coefficients wij(k) with a time delay remove the correlation between each of the outputs φ(ui(t)) of the nonlinear functions and input signals xj(t).
The restored signals ui(t) for the feedback filter structure could be expressed as follows.
[Mathematical Equation 7]
                                                                                          u                  i                                ⁡                                  (                  t                  )                                            =                            ⁢                                                                    ∑                                          k                      =                      0                                        K                                    ⁢                                                                                    w                        ii                                            ⁡                                              (                        k                        )                                                              ⁢                                          x                      i                                        ⁢                                          (                                              t                        -                        k                                            )                                                                      +                                                                                                      ⁢                                                ∑                                      j                    =                    1                                    N                                ⁢                                                      ∑                                                                  j                        ≠                                                  i                          ⁢                                                                                                          ⁢                          k                                                                    =                      1                                        K                                    ⁢                                                                                    w                        ij                                            ⁡                                              (                        k                        )                                                              ⁢                                                                  u                        j                                            ⁡                                              (                                                  t                          -                          k                                                )                                                                                                                                                    [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          7                ]            
In this structure, there exists three different cases of filter coefficients. More specifically, they are the coefficients with no time delay between the input and output wii(0), the coefficients with a time delay wii(k), k≠0 and the differential feedback filter coefficients wij(k),i≠j. The training methods for each of the cases are as follows.
                                          Δ            ⁢                                                  ⁢                                          w                ii                            ⁡                              (                0                )                                              ∝                                    1              /                                                w                  ii                                ⁡                                  (                  0                  )                                                      -                                          φ                ⁡                                  (                                                            u                      i                                        ⁡                                          (                      t                      )                                                        )                                            ⁢                                                x                  i                                ⁡                                  (                  t                  )                                                                    ,                                  ⁢                              Δ            ⁢                                                  ⁢                                          w                ii                            ⁡                              (                k                )                                              ∝                                    -                              φ                ⁡                                  (                                                            u                      i                                        ⁡                                          (                      t                      )                                                        )                                                      ⁢                                          x                i                            ⁡                              (                                  t                  -                  k                                )                                                    ,                                  ⁢                              Δ            ⁢                                                  ⁢                                          w                ij                            ⁡                              (                k                )                                              ∝                                    -                              φ                ⁡                                  (                                                            u                      i                                        ⁡                                          (                      t                      )                                                        )                                                      ⁢                                          u                j                            ⁡                              (                                  t                  -                  k                                )                                                    ,                                  ⁢                              φ            ⁡                          (                                                u                  i                                ⁡                                  (                  t                  )                                            )                                =                                                    ∂                                  P                  ⁡                                      (                                                                  u                        i                                            ⁡                                              (                        t                        )                                                              )                                                                              ∂                                                      u                    i                                    ⁡                                      (                    t                    )                                                                                      P              ⁡                              (                                                      u                    i                                    ⁡                                      (                    t                    )                                                  )                                                                        [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          8                ]            
The coefficients wii(0) control the size of data in order to maximize the amount of information which is transferred through a nonlinear function. The other coefficients wii(k),k≠0 with a time delay produce output signals corresponding to whitening of each of the input signals with respect to time. The coefficients wij(k),i≠j remove the correlation between each of the outputs φ(ui(t)) of the nonlinear functions and input signals xj(t).
FIG. 1 and FIG. 2 represent a feedforward and feedback filter structures in which each structure consists of two input and two output terminals, however, the total number of input and output terminals for each structure could be increased in order to construct a filter structure with an arbitrary number of input and output terminals. For the filter structure with an arbitrary number of input and output terminals, the same mathematical equations as represented in mathematical equations 4 through 8 which describe a convolution type mixed signal, restored signal and filter coefficient training method could equally be applied.
The restored sound source signal for the convolution type mixed signal and original sound source signal appears as whitened with respect to time. This whitening problem could be resolved by leaving the filter with the coefficients wii(0) between the inputs and outputs with the same index and fixing the value of the coefficients wii(k), k≠0 with a time delay to zero.
In case when pure noises can be observed, the independent component analysis could be utilized as an adaptive noise filter where the noise component of the mixed signal is removed.
FIG. 3 shows a diagram which illustrates the structure of a conventional adaptive noise filter.
As shown in FIG. 3, a signal source S(10) is transferred to a sensor via a channel, and a mixed signal with noise s+n0 which is generated in the sensor by addition of a noise source n0 (20) forms a primary input (30) for the noise filter.
A secondary sensor receives a noise n1 via a channel which is different to that of the first sensor, it forms a reference input (40) with respect to the noise filter.
The noise n1 is transformed into an output Z which is nearest to n0 through an adaptive filter (50) and a system output u=s+n0−Z (70) of the noise filter is formed by deducting the output Z from the primary input s+n0.
The purpose of the conventional adaptive noise filter is to obtain an output u=s+n0−Z which is nearest to the signal S from the perspective of least square. In order to achieve this object, the filter is adapted with the Least Mean Square algorithm (LMS) to minimize the total outputs of the noise filter. More specifically, the outputs of the adaptive noise filter act as an error signal for the duration of the adaptive process.
This coefficient training of the filter follows the LMS algorithm (Widrow-Hoff) and could be expressed as follows.
                              Δ          ⁢                                          ⁢                      w            ⁡                          (              k              )                                      ∝                              u            ⁡                          (              t              )                                ⁢                                    n              1                        ⁡                          (                              t                -                k                            )                                                          [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          9                ]            
where, w(k) represents a kth filter coefficient and t is a sampling number.
The conventional noise filter as shown above utilizes the LMS method as the adaptive algorithm which only uses the secondary statistical characteristic between the noise component which is to be removed and noise sources.
In order to obtain a better noise filtering performance than the LMS adaptive algorithm, the independent component analysis could be utilized.
As shown in FIG. 3 which shows a conventional adaptive noise filter, when a signal which forms the primary input, a mixed signal of noise s+n0, and a noise n1 in the secondary sensor are being received, it could be assumed that the noise n1 is independent from the signal S whereas it is related to n0 in unknown ways.
This is a reasonable assumption to be made since the noise and signal do not affect their respective sound sources. Using the independent component analysis, the noise dependent components of the first input could be removed by using the second input as following.
                                          Δ            ⁢                                                  ⁢                          w              ⁡                              (                k                )                                              ∝                                    φ              ⁡                              (                                  u                  ⁡                                      (                    t                    )                                                  )                                      ⁢                                          n                1                            ⁡                              (                                  t                  -                  k                                )                                                    ,                              φ            ⁡                          (                              u                ⁡                                  (                  t                  )                                            )                                =                                                    ∂                                  P                  ⁡                                      (                                          u                      ⁡                                              (                        t                        )                                                              )                                                                              ∂                                  u                  ⁡                                      (                    t                    )                                                                                      P              ⁡                              (                                  u                  ⁡                                      (                    t                    )                                                  )                                                                        [                  Mathematical          ⁢                                          ⁢          Equation          ⁢                                          ⁢          10                ]            
From the output of the adaptive noise filter which is obtained from the filter coefficient training, the required signal components which are independent from the noise components are obtained. The adaptive noise filter as in the case of default signal separation, the total number of input and output terminals for each structure could be increased in order to construct a filter structure with an arbitrary number of input and output terminals. For the filter structure with an arbitrary number of input and output terminals, the same mathematical equations as represented in mathematical equations 4 through 8, which describe the filter coefficient training method, could equally be applied.
However, in order to implement the adaptive filter algorithm using independent component analysis to the real world problem, a large number of filter training coefficients are required and also a large amount of calculation is required when a training is undertaken. This results in a very slow learning speed and the deterioration in the quality of result signals.